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METHOD AND APPARATUS FOR MULTIPURPOSE 
MONITORING SYSTEM 

This is an application for a patent of addition to pending application No. 
150745. 

Field of tV>ft Invention 

This invention relates to a novel use of the system described in the co- 
pending application No. 150745 of the same applicant as a passive electro- 
optical radar system for locating foreign objects in the region of a 
monitored environment. 



Backg£Quad o f * hp Invention 

In a multiplicity of environments it is desirable to prevent, ehminate or 
15 reduce the existence and/or the intervention of foreign objects. Such types 
of environment can be airport runways, military bases, home industrial 
premises, etc. A foreign object can be a person, wildlife, birds, inanimate 
objects, vehicles, fire, etc. 

20 Co-pending application No. 150745 discloses the entire content of a 
monitoring , system that prevents, eliminates or reduces the existence 
andVor the intervention of such foreign objects. Fig. 1 schematically 
illustrates the monitoring system 10, according to a preferred embodiment 
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of the co-pending application No. 150745. System 10 comprises at least one 
Charged Coupled Device (CCD) camera 12, and/or at least one thermal 
camera 11 (i.e., Infra Red camera), motors 13 and a computerized system 
15. 

The computerized system 15 is responsible for performing the processing 
required for the operation of the monitoring system. The computerized 
system 15 receives data from the cameras, processes the received data in 
order to detect, in real-time, dangerous objects at the monitored area and 
then generates data regarding the detection of suspected dangerous 
objects. 



Each camera that is attached to system 10 is rotated horizontally (i.e., 
pan) and/or vertically (i.e., tilt) by the motors 13. The rotation of the 
15 cameras is required for scanning the monitored area. Each camera 
attached to the system 10 is able to constantly scan a portion or the entire 
environment. The scanning is divided into several and a constant number 
of tracks, upon which each camera is focused. 

20 In some cases a radar system is used in order to detect and locate the 
geographical location of targets or objects in the monitored area. However, 
it is extremely desirable to perform the detection without exposing the 
activity of the radar system. 
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Hence, it is therefore an object of this invention to provide a novel method, 
implemented by the apparatus of the co-pending Israeli Application No. 
150745 for continuously and automatically detecting and finding the 
5 geographical location of dangerous objects that may constitute a menace to 
the monitored area, and this without generating a radiation. 

It is another object of this invention to provide an enhanced display of the 
detected dangerous objects. 

10 

It is yet another object of this invention to reduce the number of false 
alarms. 

Other objects and advantages of this invention will become apparent as 
15 the description proceeds. 

While the embodiments of the invention are mainly described with 
reference to application in airfields, they, of course, also can be used for 
other applications in which there exists the threat of possible intrusion of 
20 persons, dangerous objects and/or vehicles into monitored areas, which 
usually are . restricted. It is to be kept in mind that the possibility exists 
that dangerous objects may also not be natural, such as birds, but 
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artificial, used for sabotage or terror operations, or a fire endangering the 
monitored area. 

The aircraft taking off or landing on the airfield, and vehicles or persons 
5 allowed to be in the monitored area will be designated hereinafter as 
"authorized bodies". All such birds, wildlife, persons, static objects, 
artificial objects, fire and any other FODs will generally be caUed 
"dangerous objects". 

10 Summary of the Invention 

The novel method of this invention comprises: a) generating a panoramic 
image and a map of the monitored area by scanning said area, said 
sc annin g being performed by rotating at least a pair of distinct and 
identical imagers around their central axis of symmetry; b) obtaining the 
15 referenced geographical location of a detected object by observing said 
object with said pair of imagers, said geographical location being 
represented by the altitude, range and azimuth parameters of said object; 
and c) displaying the altitude value of said object on said panoramic image 
and displaying the range and the azimuth of said object on said map. 

20 

Preferably, the imagers are cameras selected from the group consisting of: 
CCD or CMOS based cameras or Forward Looking Infra Red (FLER) 
cameras. 
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The novel method of this invention implemented on apparatus which 
comprises: a) at least a pair of distinct and identical imagers for carrying 
out photographic observation of the controlled space or sections thereof; b) 
a set of motors for changing the sections of the said photographic 
observation; c) elaborator means for obtaining the referenced geographical 
location of a detected object in said controlled . space, said geographical 
location being represented by the altitude, range and azimuth parameters 
of said object; d) means for generating a panoramic image and a map of 
the monitored area; e) means for displaying the altitude value of said 
object on said panoramic image and means for displaying the range and 
the azimuth of said object on said map. 

Brief Description of the Drawings 
In the drawings: 

- Fig. 1 schematically illustrates a runway monitoring system, 
according to a preferred embodiment of the co-pending Israeli 
Application No. 150745; 

- Fig. 2 schematically illustrates the solving of the general three 
dimensional position of an object in the Y direction; 

- Fig. 3 schematically illustrates a combined panoramic view and map 
presentation of a monitored area; 
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Fig. 4 schematically illustrates 
vertical rotation axis; and 
Fig. 5 schematically illustrates 
horizontal rotation axis. 



a scanning of a sector around a 



a sc annin g of a sector around a 



Detailed Description of Preferr ed Embodiments 

All the processing of this invention is digital processing. Grabbing an 
image by a camera, such as those of the apparatus of this invention, 
generates a sample image on the focal plane, which sampled image is 
preferably, but not limitatively, a two-dimensional array of pixels, wherein 
to each pixel is associated a value that represents the radiation intensity 
value of the corresponding point of the image. For example, the radiation 
intensity value of a pixel may be from 0 to 255 in gray scale, wherein 0 = 
black, 255 = white, and others value between 0 to 255 represent different 
levels of gray. The two-dimensional array of pixels, therefore, is 
represented by a matrix consisting of an array of radiation intensity 
values. 

Hereinafter, when an image is mentioned, it should be understood that 
reference is made not to the image generated by a camera, but to the 
corresponding matrix of pixel radiation intensities. 
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Each sampled image is provided with a corresponding coordinates system, 
the origin of which is preferably located at the center of the sampled 



image. 



5 In this invention, the determination of the background space, the objects 
evaluation programs, the extrapolation of the monitored paths of 
dangerous objects, the estimation of possible dangers of collision and the 
actions for ehminating such danger, is the same as that described in the 
co-pending Application No. 150745. Furthermore, the taken sequence of 

10 images of the monitored environment, as well as the detection of moving 
and/or static objects at the pixel processing stage, is also the same as that 
described in the co-pending Application No 150745. 

According to a preferred embodiment of the present invention, each 
15 camera attached to the system 10 constantly scans a portion of the 
environment or its entirety. For a typical camera model (e.g., Raytheon 
commercial infrared series 2000B controller infrared thermal imaging 
video camera, of Raytheon Company, U.S.), which is suitable to be 
attached to system 10, it takes about 15 seconds to scan the complete 
20 runway environment that is covered by it. The scanning is divided into 
several and a constant number of tracks, upon which each camera is 
focused. The preferred scanning area is preformed at the area ground up to 
a height of, preferably but not limitatively, two hundred meters above the 
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area ground and also at a distance of a few kilometers, preferably 1 to 2 
Km, towards the horizon. 

Preferably but not limitatively, the cameras of system 10 are installed on a 
5 tower (e.g., flight control tower) or on other suitable pole or stand, at a 
height of between 25 to 60 meters above the desired monitored area 
ground. 

The cameras can be positioned in a variety of ways. According to one 
10 preferred embodiment of this invention, a pair of identical cameras is 
located vertically one above the other on the same column, so that the 
distance between the cameras is approximately between 1 to 2 meters. The 
stand on which the cameras are located can be rotated by a motor, thus on 
each turn of the stand, both of the cameras are moved together 
15 horizontally. In such a configuration the cameras scan a sector, track or 
zone simultaneously. Preferably, but not hmitatively, the distance between 
a pair of cameras is between 0.5 to 50 meter, horizontally, vertically or at 
any angle. 



20 According to a preferred embodiment of this invention, system 10 (Fig. 1) 
is used as a system for detecting targets and their geographical location 
and' this without generating radiation (i.e., a passive electro-optical radar). 
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Preferably, the geographical location of the targets is given in polar 
coordinates, e.g., range and azimuth. 

In this invention, system 10 is used to measure and provide the 
geographical location (i.e., the location of the object in a three-dimensional 
coordinates system) of a detected object, such as the range, azimuth and 
altitude of the object. The geographical location is relative to a reference 
coordinates system on earth. The location of the object in the three- 
dimensional coordinates system is obtained due to an arrangement of at 
least two imagers, as will be described hereinafter. Preferably, the imagers 
are digital photographic devices such as CCD or CMOS based cameras or 
Forward Looking Infra Red (FLIR) cameras. 

Preferably, at least a pair of identical CCD cameras, such as camera 12 of 
Fig. 1 and/or pair of FLIR cameras, such as camera 11 of Fig. 1 are 
positioned in such a way that system 10 sees each object, as it is captured 
by the charged coupled device of each camera, in two distinct projections. 
Each projection represents an image that comprises a segment of pixels 
wherein the center of gravity of a specific object in the image has specific 
coordinates, which differ from its coordinates in the other projection. The 
two centers. of gravity of the same object have the pixel coordinate system 
(xl, yi) for the first camera and the pixel coordinate system (x2, y2) for the 
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second camera (e.g., each coordinate system can be expressed in units of 
meters). 

System 10 essentially comprises at least two cameras preferably having 
parallel optical axes and having synchronous image grabbing. A rotational 
motion means such as motor 13 and image processing means, as described 
in the co-pending Application No. 150745. The image processing means is 
used to filter noise-originated signals and extract possible targets in the 
images and determine their azimuth, range and altitude according to their 
location in the images and the location disparity (parallax) in the two 
images coming from the two cameras (e.g., two units of CCD camera 12). 

Obtaining the general location of an object in an image is identical for both 
directions X and Y of the coordinates system. Fig. 2 schematically 
illustrates the solving of the general three-dimensional position of an 
object in the Y direction. 

Thus, solving the coordinate for the three-dimensional coordinates system 
is obtained as follows: 

At first, the two following equations are provided, 
(1) * 
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y L= I L 

f Zi 

(2) 



solving for Zl and Yl, we get: 



(3) 



Z - D * f 
' Ay Ay = y\-y2 



(4) Y >-J Z >- Ay 



1 0 and the same for XI : 



V X l* D 

(5) Xl=S J Z, ~~aT 



wherein, 

15 D - distance between the cameras optical axes; 
f - focal length of the camera lenses; 

(xl, yl) - coordinates of the target projection onto the first camera ■ 
array; 

(x2,sy2) - coordinates of the target projection onto the second 
20 detector array; 
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(X1,Y1,Z1) - coordinates of the target in the local coordinate system; and 
(X,Y,Z) - coordinates of the target in the general world coordinate system 

Due to the fact that the system 10 is scanning with the two cameras a 
5 certain sector, each scan step has a certain azimuth angle a which is 
dissimilarity with the system initial position. .The system initial position 
represents the general world coordinate system. The magnitude of the 
angle a is used for correcting the dissimilarity by rotating the local step 
coordinates system thus that it will match the general world coordinate 
10 system. 



In other words, the coordinates of an object in the local coordinate system 
differ from the coordinates of that object in the general world coordinate 
system. Thus, the transformations from the local coordinate system to the 
1 5 general world coordinate are calculated as follows: 

X = X,*cosa-Z,* since 
Y = Y1 

Z = X t *sina + Z, *cosa 

20 

This covert detection and localization of dangerous objects embodiment 
provides a passive operation of system 10 by imaging optical radiation in 
the far infrared range that is emitted by the relatively hot targets, such as 
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an airplane, helicopter, boat, a human being or any other object. This 
embodiment further provides a passive operation of system 10 by imaging 
optical radiation in the near infrared or vision ranges that is reflected by 
said targets. 

5 

In this embodiment, system 10 generates, by elaborator means, a 
panoramic image of the scene (i.e., of the monitored area) by rotating the 
pair of cameras around their central axis of symmetry, as well as a map of 
the detected targets in the scene that is regularly refreshed by the 

10 scanning mechanism of system 10. The combination of a panoramic image 
aligned with a map of the detected targets (i.e., dangerous objects) form a 
three-dimensional map of the targets, as shown in Fig. 3. Preferably, the 
elaborator means consisting of the computerized system 15 and a 
dedicated algorithm installed it, as will known to a person skilled in the 

15 art. 

Reduction of the number of false alarm is also achieved by the reduction of 
clutter from the radar three-dimensional map. This is done, as has already 
been described in co-pending application No. 150745, by letting system 10 
20 assimilate the surrounding response, coming from trees, bushes, vehicles 
on roads and the like and reducing the system response in these areas 
accordingly, all in an effort to reduce false alarms: 
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System 10 scans the monitored area by a vertical and/or horizontal 
rotational scanning of the monitored area. The vertical rotational sc annin g 
is achieved by placing the system axis of rotation perpendicular to the 
earth and the sc annin g is done over the azimuth range, which is the same 

5 as that done in typical radar scanning. The horizontal rotational scanning 
is achieved by placing the system axis of rotation horizontal to the earth 
and the scanning is done over elevation angles. These two last distinctions 
are needed in different situations in which the target exhibits certain 
activities that call for such scanning. Of course, by adding more than two 

10 imager means (e.g., such as three or four CCD cameras), the accuracy of 
the range measurement is increased. 

Fig. 3 schematically illustrates a combined panoramic view and map 
presentation of a monitored area. In Fig. 3, the electro-optical radar (i.e., 

15 system 10) is sc annin g with a viewing angle confined by the two rays, 20 
and 30. The radar display is arranged in a graphical map presentation, 40, 
and a panoramic image 50. In the map, the relative locations of the 
targets, 60 and 70, can be seen, while in the panoramic image, 50, the 
heights of the targets can be seen. The displayed map and panoramic 

20 image are both refreshed with the radar system rotational scanning. The 
combination of a panoramic view, providing altitude and azimuth, with a 
map, providing azimuth and range, gives a three-dimensional map of 
targets. Preferably, the geographical position of each detected object being 
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displayed by using any suitable three-dimensional software graphics, such 
Open Graphic Library (OpenGL), as known to a skilled person in the 



as 
art 



10 



Using two FLIR cameras positioned on the system vertical axis and two 
additional video cameras (e.g., CCD cameras), operating in the normal 
vision band, located horizontally from the two sides of the system vertical 
axis, the different camera types are optimal on different conditions: the 
FLIRS are optimal at night and in bad weather and the video cameras are 
optimal in the daytime and in good weather. 



In Fig. 4, the pair of cameras 12 of the electro-optical radar embodiment of 
system 10 is rotating around the vertical rotation axis 80 and providing an 
image of scene, which is confined between the rays 100, 110, 120 and 130. 
15 The provided image of the scene is analogous to a radar beam, thus while 
the cameras are rotating around axis 80, the beam is scanning through the 
entire sector 135. 

In Fig. 5, another scanning option is introduced in which the cameras 12 of 
20 the electro-optical radar (i.e., system 10) are rotating around the 
horizontal rotation axis 140, thereby scanning sector 160. Preferably, the 
scanning of this sector 160 is performed by the same method as the 
vertical scanning. 
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According to this embodiment of the present invention, the distance of the 
targets is measured by using radiation emitted or reflected from the 
target. The location of the target is determined by using triangulation with 

5 the two cameras. This arrangement does not use active radiation emission 
from the radar itself and thus remains concealed while in measurement. 
The distance measurement accuracy is directly proportional to the pixel 
object size (the size of the pixel in the object or target plane) and to the 
target distance and inversely proportional to the distance between the two 

10 cameras. The pixel size and the distance between the cameras are two 
system design parameters. As the distance between the two cameras 
increases and the pixel size decreases, the distance measurement error 
decreases. 



15 Another feature of this embodiment is the ability to double-check each 
target detected, hence achieving a reduction in the number of false alarms. 
The passive operation allows a reliable detection of such targets with a 
relatively low false alarm rate and high probability of detection by 
utilizing both CCD and/or FLEEt cameras to facilitate double-checking of 

20 each target detected by each camera. Each camera provides an image of 
the same area but from a different view or angle, thus each detected target 
at each image from each camera should be in both images. As the system 
geometry is prior knowledge, hence the geometrical transformation of one 



10 
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image to the other image is known, thus each detected pixel in one image 
receives a vicinity of pixels in the other image, and each of them may be its 
disparity pixel. Thus only a pair of such pixels constitutes a valid 
detection. 

From the above description of the system scanning methods, the system 
display of detected targets may include all the measured features, e.g., 
target size, distance from the system, azimuth, and altitude. The present 
invention uses a panoramic image of the scene together with its map of 
detected targets to present the above features, in a convenient and concise 
manner. 



The above examples and description have of course been provided only for 
the purpose of illustration, and are not intended to limit the invention in 
1 5 any way. As will be appreciated by the skilled person, the invention can be 
carried out in a great variety of ways, employing more than one technique 
from those described above, all without exceeding the scope of the 
invention. 



15212/02 




CLAIMS 

1 . Method for the monitoring of an environment, comprising: 

a) generating a panoramic image and a map of the monitored 
area by scanning said area, said scanning being performed by 
rotating at least a pair of distinct and identical imagers around 
their central axis of symmetry; 

b) obt ainin g the referenced geographical location of a detected 
object by observing said object with said imagers, said geographical 
location being represented by the altitude, range and azimuth 
parameters of said object; and 

c) displaying the altitude value of said object on said panoramic 
image and displaying the range and the azimuth of said object on 
said map. 

2. Method according to claim 1, wherein the imagers are photographic 
devices selected from the group consisting of: CCD or CMOS based 
cameras or Forward Looking Infra Red (FLEEt) cameras. 

3. Apparatus according to claim 1, in which the distance, in an angle, 
between each two imagers is between 0.5 to 50 meters. 

4. Apparatus for the monitoring of an environment, which comprises: 
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a) at least a pair of distinct and identical imagers for carrying 
out photographic observation of the controlled space or sections 
thereof; 

b) a set of motors for changing the sections of the said 
photographic observation; 

c) elaborator means for obtaining the referenced geographical 
location of a detected object in said controlled space, said 
geographical location being represented by the altitude, range and 
azimuth parameters of said object; 

d) means for generating a panoramic image and a map of the 
monitored area; 

e) means for displaying the altitude value of said object on said 
panoramic image and means for displaying the range and the 
azimuth of said object on said map. 

5. Apparatus according to claim 4, in which the imagers are 
photographic devices selected from the group consisting of: CCD or 
CMOS based cameras or Forward Looking Infra Red (FLER) 
cameras. 

6. Apparatus according to claim 4, in which the distance, in an angle, 
s between each two imagers is from 0.5 to 50 meters. 
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^ 7. Apparatus according to claim 4, in which the means for displaying 
the monitored area are using three-dimensional software graphics 
where the geographical location of each detected object is indicated 
as a three-dimensional image. 
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